1. Field of the Invention
The invention relates to a method and device for furnishing motion vectors, and to an image signal processing apparatus and an image display apparatus incorporating such a motion vector processing device.
2. Description of the Related Art
German offenlegunsschrift DE-A-195.10.389 discloses a method and a circuit arrangement for receiving coded video signals. The method concerns a video signal which contains a sequence of images with compressed redundancy, e.g., coded in accordance with the MPEG-2-standard. The motion decoding is carried out first. The post-processing of the motion-decoded fields, this post-processing essentially containing interpolations, is carried out next, together with processing to remove flicker. Advantageously, the motion vectors retrieved at the motion decoding, are used also for the flicker removal.
Motion estimation can be seen as an optimization problem: a sometimes complex, but usually simple criterion function has to be minimized or maximized to find an output motion vector. There are brute force methods that simply try all possible vectors, in a predefined range, in order to be sure to obtain the global optimum of the criterion function. Also, there are efficient approaches that test only the most likely motion vectors. This likelihood is usually determined by spatial or temporal proximity, and, consequently temporal and spatial prediction vectors have been popular in the efficient motion estimation algorithms. Depending on the motion estimation algorithm used, the properties of the resulting motion vector fields are different, so that the application areas of the various motion estimation algorithms are different. Motion vectors can be used in high quality video signal processing, like motion compensated scan conversion, motion compensated noise reduction and image coding. The intended processing puts constraints on various quality aspects of the motion vectors. For example, high quality motion compensated scan conversion requires a motion vector field where the vectors represent the true motion vectors (i.e., true direction and speed) within the image sequence. For coding applications, this is irrelevant, as the only quality criterion is a low average prediction error.
Brute force methods, like full search block matching, tend not to give true motion vectors. Instead, full search block matching minimizes the error between blocks, so its vectors will connect two blocks that have a minimal error between these two blocks, which is suitable for minimizing the prediction error in coding applications. While these minimal errors work efficiently for compression algorithms, like MPEG, the resulting vectors are unsuitable for use in high quality scan conversion. On the other hand, more efficient motion estimators, like 3-D recursive block matching, tend to give true motion vectors that can be used in high quality scan conversion, but which may not be suitable for use in compression algorithms.
With the emergence of MPEG and other types of video compression, it is possible to have input video material that already contains motion vectors, so, at first sight, it may seem that for this type of source material, motion estimation at the receiver side is not necessary. However, it is not certain that these motion vectors can be directly used for, e.g., scan rate conversion, because it is unknown whether the MPEG motion vectors represent true motion or not. At the decoder/receiver side, it is unknown what type of motion estimation algorithm was used at the encoder/transmitter side, so one must assume, as a worst case situation, that the MPEG motion vectors are optimized for an efficient compression, and that they do not represent true motion vectors. For example, a homogeneous background could produce strange and long inconsistent vectors when small changes of brilliance happen in a heterogeneous way. More specifically, periodical structures and noise in picture areas with little detail, may cause such inconsistent vectors. Another problem with motion vectors in MPEG data streams, is that it is uncertain that all motion vectors are transmitted within the data stream. For these reasons, the technique proposed by DE-A-195.10.389 seems not to be suitable for high quality post-processing.
In an attempt to solve a similar problem, it has been proposed to smooth the vectors transmitted with a digitally encoded television signal before using these vectors for a field rate upconversion operation, see the M.S.E.E. graduation report "Bewegingsgecompenseerde interpolatie van digitale televisiebeelden", by L. J. Steenbeek, Delft University of Technology, December 1986, pp. 60-68.